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ABSTRACT 

Efforts to assess institutional effectiveness not 
only enable community colleges to meet accreditation mandates, but 
can also serve as a catalyst for institutional renewal. Institutional 
effectiveness has become an important topic for the 1990s as a result 
of past neglect of accountability, new legislative mandates for 
education, changes in accreditation criteria from process-oriented to 
outcomes-oriented, and a renewed interest on the part of colleges in 
improving the quality of instruction. To assess institutional 
effectiveness a college must define the mission of the college; 
articulate the major results that are expected from the achievement 
of the mission; and determine the specific evidence that will be 
acceptable to determine whether those results have been achieved. At 
a minimum, institutional assessment processes will require that 
institutions: (1) articulate their mission; (2) establish a planning 
mechanism; (3) develop an evaluation system; (4) identify critical 
areas of success; (5) establish priority standards upon which the 
college can judge its effectiveness; (6) determine mechanisms for 
documenting if the established standards have been met; and (7) 
utilize the results of assessment for decision making. In addition, 
colleges must enlist the support of the president and board of 
trustees, involve all units of the college, and determine how to pay 
for assessment. Indicators of effectiveness might include transfer 
student success, job placement rates of graduates, employer 
oatisf action, and economic impact, of the institution. Institutions 
should publicly recognize individuals and departments that make 
significant contributions toward achievement of the college's 
mission. (PAA) 
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INSTITUTIONAL EFFECTIVENESS: 
A STRATEGY FOR INSTITUTIONAL RENEWAL 



Emphasis on accountability and effectiveness offers a perfect opportunity 
for community colleges to gain much desex'ved regard from their communities and 
constituents . For too long community colleges have been judged by university 
standards. T n the October 15, 1990 U.S. News and World Report list of 
America's best colleges , community colleges were conspicuously absent . Several 
authors in the past decade judged community colleges' effectiveness by 
university standards or by very narrow criteria related to the transfer function . 
The problem is not that the standards are too high - they are often not the 
appropriate standards . Most effectiveness and accountability systems measure 
the degree to which an organization is successful in carrying out its stated 
purpose . Community colleges have a unique mission and should be measured by 
standards relevant to their mission. The leadership of community colleges must 
seize the institutional effectiveness initiative as an opportunity to shape an 
agenda that demonstrates the success of the "people's college" in achieving its 
mission. 

Institutional effectiveness processes not only enables two-year colleges to 
meet the minimum standards of accreditation and state mandates, they can 
become a catalyst for institutional renewal. Three basic questions about 
institutional effectiveness will be addressed in this paper. 

1 . Why hzs institutional effectiveness become a central agenda item for 
liigher education in the 1990s? 

2. What is the meaning and application of effectiveness to the 
community college? 

3. How does a college initiate a workable program of institutional 
effectiveness? 
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The WHY of Institutional Eff ectiyenggg 

Past neglect of accountability is one obvious reason why institutional 
effectiveness has become the hottest new topic on the educational agenda. 
Colleges have tended to say to their constituents and to their funding agencies - 
- "Trust us. What we do cannot be measured." — or — "Our work is beyond 
your comprehension." Evidence is mounting that such tactics are no longer 
acceptable. 

In the 1990s, educators must step up to the line and become more 
accountable for the almost 400 billion dollars spent annually on education in 
America. 

In December 1989, B. Franklin Skinner, President of the Southern Bell 
Corporation, addressed a general session of the Southern Association of Colleges 
and Schools on the topic "Two Cheers for Higher Education." 1 He withheld the 
third cheer because, in his opinion, higher education has failed to keep its 
mission in clear focus . He used this quote from Peter Drucker to make his point, 
"The present level of rigor and narrow focus of undergraduate education is not 
preparing graduates to help business compete in the international marketplace . " 

Skinner shared a story about Lawrence of Arabia to make his point. The 
Arabian leader took several Arab chieftains to peace talks in Paris. They were 
awed by the modern wonders they had never seen before , especially the brass 
faucets in their hotel rooms . These men of the desert were fascinated that water 
could be made to flow by a flick of the wrist. When they returned to the Middle 
East, one of the men had removed a faucet and brought it along fully expecting 
that water would flow out of it in the Arabian desert. 

In the past, business and legislative leaders have been in such awe of 
higher education's golden domes and ivy covered walls (our brass faucets) that 
they have been reticent to critique the effectiveness of our services and 
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programs. However, in 1990, we face a new day. According to Mr. Skinner, 
those who pay the bill are vitally interested in what comes out of the faucet. 

Political initiatives are another reason for the focus on 
effectiveness. Because higher education, in large measure, has elected not to 
be accountable, legislatures in most states have initiated accountability 
mandates. According to Ted Marchese, Vice President of the American 
Association of Higher Education (AAHE), only five states had accountability 
mandates in 1985. In 1990, forty states had such mandates and several of the 
remaining ten have accountability measures under consideration. 

The change in regional accreditation criteria is the most practical reason 
for the new attention to institutional effectiveness . When the North Central 
Accrediting Association revised its standards in February 1990 to include 
institutiontl effectiveness measures, it was the last of the six regional 
accrediting bodies to demonstrate a commitment to outcomes oriented 
accreditation. The Commission on Colleges of the Southern Association of 
Colleges and Schools ( SACS ) which accreditates southern community colleges has 
been the vanguard leader in the educational reform movement. In 1984 SACS 
moved from process- to outcomes-oriented accreditation. Process-oriented 
accreditation measures quality in terms of enrollment, volumes in the library and 
credential of faculty, which may or may not have anything to do with the quality 
of the educational experience . Outcomes -oriented accreditation measures quality 
in terras of student learning and success and addresses the question "What 
impact has the college had upon the individual student or the community it 
serves?" Ready or not, community colleges must deal with institutional 
effectiveness as they seek to reaffirm their accreditation. 

In addition to the reasons outlined above , institutional effectiveness has 
been embraced by many colleges because it improves the quality of teaching and 
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learning and provides an opportunity to communicate past and present 
effectiveness. Institutions who have made a college-wide commitment to 
institutional effectiveness are willing to ask tough questions about how much 
their students are learning. When colleges focus on the content and the outcome 
of courses and programs, the teaching and learning process is enhanced. 
Further, many community colleges have learned that an outcomes-oriented 
evaluation process produces data that demonstrate a high degree of satisfaction 
with college programs and services . 

Peter T . Ewell of the National Center for Higher Education Management in 
his book, Assessing Institutional Effectiveness , observes "The underlying goal 
of any organization is to improve effectiveness."" The national emphasis on 
accountability provides the paradigm in which this goal of the community college 
can be realized. 

The WHAT of Institutional Effectiveness 

The definitions of institutional effectiveness range from the simple to the 
complex . Some argue that institutional effectiveness is just a new way of looking 
at planning and evaluation — "old wine in new wine skins." Historically, 
colleges have had offices of planning and offices of institutional research. 
However, too often planning has not been driven by a serious in-depth 
evaluation of the strengths and weaknesses of the college; nor have the results 
of institutional research been used to measure the effectiveness of planning. 
The Commission on Colleges (SACS) Criteria says, "Institutions have an 
obligation to all constituents to evaluate effectiveness and to use the results in 
a broad base continuous planning and evaluation process." 3 

Peter Ewell, who assisted the Commission on Colleges in the development 
of its Criteria, defines "Institutional effectiveness as the comparison of the 
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results achieved to the goals intended."' 4 The SACS Criteria concludes that 
institutional effectiveness is matching performance to purpose. The second 
"must" statement in Section III of the Criteria reads "All institutions must define 
their expected educational results and describe how the achievement of these 
results will be ascertained." 

Perhaps the most thorough definition of effectiveness has been provided 
by the National Alliance of Community and Technical Colleges — "Institutional 
effectiveness is the process of articulating the mission of the college , setting 
goals , defining how the college and community will know when these goals are 
being met and using the data from assessment in an on-going cycle of planning 
and evaluation." 15 The Council on Post- Secondary Accreditation makes the 
following statement about institutional effectiveness, "The quality of an 
educational process is related to (1) the appropriateness of its objectives; (2) 
the effectiveness of the use of resources in pursuing these objectives; and (3) 
the degree to which these objectives are achieved. Without a clear statement of 
what education is expected to provide, it is not possible to determine how good 
itis." e 

A review of these definitions and observations about institutional 
effectiveness suggests that while the accountability movement incorporates many 
of the established educational processes and academic departments of colleges , 
it involves a new commitment at the governing board and executive 
administration level to assure that college programs and resources are more 
effectively used to achieve the institution's purpose. 

The HOW of Institutional Effectiveness 

Accreditation criteria and state mandates allow considerable flexibility in 
how an institution organizes and implements a program of institutional 
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ef fectiveness . In the October 1990 edition of Change Magazine , Russell Edgerton 
outlined the broad scope of the assessment process when he observ.ed, "It's a 
mindset that asks questions — good questions, hard questions, legitimate 
questions — about what and how much our students are learning." 7 For most 
colleges it involves asking and answering three basic questions. 

1. What is the mission (business) of our college? 

2 . What arc the major results we expect from the achievement of the 
mission? 

3 . What specific evidence are we willing to accept that these results 
' have been achieved? 

The following model of institutional effectiveness developed by the National 
Alliance of Community and Technical Colleges outlines the basic components of 
an institutional effectiveness program. 



The Model of Institutional Ef'ectiveness 
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While the process for developing or operationalizing institutional 
effectiveness varies, it normally contains seven basic steps. 

1 . Articulate the mission 

2 . Establish a planning mechanism 

3 . Develop an evaluation system that tells if "the college is doing what 
it says it does" 

4 . Identify critical areas of success 

5 . Establish priority standards upon which the college can judge its 
effectiveness in the identified critical areas 

6 . Determine mechanisms for documenting if the established standards 
have been met . 

* Hard data (enrollment .eports, licensure test results, 
transfer grades , assessment of majors , etc . ) 

* Surveys (written, telephone, interviews) 

* Peer reviews 

7 . Utilize results of assessment for decision making 

In addition to these basic organizational steps , a college must consider the 
following related issues: 

1 . Support of the president and board of trustees 

2. Linkage of assessment to the college mission 

3. Involvement of all units of the college 

4 . Emphasis on quality throughout the college , with assessment viewed 
as supporting quality decisions 

5 . Establishment of an organizational structure to monitor and report 
on effectiveness outcomes 

6 . Determination of how to pay for assessment activities 

ERIC 9 
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A basic element of an institutional effectiveness program is the 
establishment of indicators of effectiveness — data that document that the 
college has achieved his mission and goals and that there are identifiable key 
factors that must go right if the organization is to achieve its mission. The Sloan 
School of Business of MIT developed an evaluation process for business and 
industry call Critical Success Factors (CSF) . The CSF process suggests that 
everything a business does is not of equal value . A business should determine 
the activities most critical to its success and focus emphasis and resources on the 
critical areas. 8 

This process of focusing on mission-related issues can be applied to 
educational institutions . DeAnza College in California and Midlands Technical 
College in South Carolina havp adapted the Critical Success Factors model at the 
college level and the North Carolina Board of Community Colleges has applied the 
concept at the system level. For example, Midlands Technical College has 
identified six critical success factors and twenty indicators of effectiveness. 
Institutional effectiveness measures utilized by colleges might 

include : 

success of transfer students at senior institutions 
placement rates of graduates 
satisfaction of employers 

satisfaction of students with. programs and services 
impact of the institution on economic development 
When these data are presented to constituents, governing boards and 
funding agencies, they have generally led to an enhanced reputation of the 
effectiveness of the two-year community college . 

Perhaps more important than identifying the indicators of effectiveness 
and gathering data to demonstrate the achievement of the college's educational 
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goals is how institutions , their governing boards and governmental agencies use 
these data . Philip B . Crosby in his book Quality is Free observed , " It isn' t what 
you find, it's what you do with what you find. MO To gain college-wide support 
of institutional effectiveness and to enhance the work climate of the institution, 
institutional effectiveness data must first be used for celebration — then for 
correction. Kouzes and Posner in their book The Leadership Challenge argue 
"Cheerleading is a large part of the leader's function. Leaders are always on the 
lookout for people who are doing the right thing and in the right way so they can 
celebrate victory." 10 

Most colleges that have had extended experience with institutional 

i 

effectiveness are pleased that the findings generally demonstrate educational 
results that bring commendation to the institution. The institution should look 
for ways to recognize individuals and departments that make significant 
contribution toward the achievement of the college's mission. However, when 
data demonstrates excessive cost, undesirable outcomes and lack of correlation 
between resources and college mission, college leaders must be courageous 
enough to act. Otherwise, the planning and evaluation process will lack 
credibility. 

Admittedly, institutional effectiveness is a time consuming and costly 
enterprise. In spite of the considerable investment of personnel and resources 
required of most institutions, the choice is no longer optional. Therefore, 
institutions should initiate a structured program of institutional effer *iveness at 
the earliest possiMe time and look for practical outcomes which include: 

* Clarification of mission 

* Improved use of resources 

* Identification of priorities 

* Improved performance 
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* Increased return on resources invested 

* Enhanced reputation 

* Energized work environment 

In addition to these practical applications, institutional effectiveness 
should be a catalyst for institutional renewal . Dr . Albert L . Lorenzo , President 
of Macomb Community College, has written extensively on the parallels between 
organizational development theory and the application of institutional 
effectiveness. He concludes "Effectiveness is not a measurement process, it's 
a change process. 1,11 He describes the well-known life cycle theory of any 
organization that includes birth, a period of energetic and phenomenal growth, 
and maturation, followed by either a decline or renewal based on the introduction 
of a change agent. Dr. Lorenzo argues that institutional effectiveness can be 
that change agent for renewing the American community college . 

The Resource Manual on Institutional Effectiveness of the Commission on 
Colleges of the Southern Association of Colleges and Schools concludes "A self- 
evaluating organization has been described as an organization constantly in 
conflict with itself. Such tensions are worth enduring only if as a result, 
institutions overcome their resistance to change and provide positive incentives 
for faculty members and administrators alike to become involved in using 
evaluation results to improve programs and services. " ia 

As a college initiates an effectiveness program, it should be remembered 
that the main purpose of evaluation is to improve the college for the benefit of 
its students. Community colleges should seize this opportunity to define 
effectiveness, and use the results to improve programs and services and 
celebrate our successes. 



12 



9 



r 



ERIC 



REFERENCES CITED 

1. Skinner, B. Franklin. (December 11, 1988). "Two Cheers for Higher 
Education" . Speech to the Southern Association on Colleges and Schools . 
Atlanta, Georgia. 

2. Ewell, Peter T. and Lisensky, Robert P. (1988). Assessing 
Institutional Effectiveness: Redirecting the Self -Study Process . 
Consortium for the Advancement of Private Higher Education . NCHEMS . 
Balden, Colorado. Page 19. 

3. Southern Association of Colleges and Schools. (1991 Edition) . Criteria for 
Accreditation . Atlanta, Georgia. Page 14. 

4 . Ewell , Peter T . ( 1983 ) . Information on Student Outcomes : How to Get it and 
How to Use it . NCHEMS. Bolder, Colorado. Page 89. 

5. Grossman, Garry M. and Duncan, Mary Ellen. (1989). Indications of 
Institutional Effectiveness : A Process for Assessing Two-Year Colleges . 
Center on Education and Training for Employment. Columbus, Ohio. 
Page 5 . 

6. Southern Association of Colleges and Schools. (1989) . Res urce Manual on 
Institutional Effectiveness . Atlanta, Georgia. Page 19. 

7. Edgerton, kussell. (October, 1990). "Assessment at Half-time . " Change . 
Page 5 . 

8 . Bullen , Christine and Rockart , John . ( 1 981 ) . A Prime on Critical Success 
Factors. Center for Information Systems Research. Massachusetts 
Institute for Technology. Camoridge, Massachusetts. 

9. Crosby, Phillip B. (1979). Quality is Free . New American Library. New 
York, New York. Page 65. 

10. Kouzes, James M. andPosner, Barry Z. (1988). The Leadership Challenge . 
Jos sey Bass. San Francisco, California. Page 26. 

11 . Lorenzo, Albert L. (April 27 , 1990) . Presentation at the Annual Meeting of 
the American Association of Community and Junior Colleges. Seattle, 
Washington. 

12. Southern Association of Colleges and Schools. (1989) . Resource Manual on 
Institutional Effectiveness . Atlanta, Georgia. Page iv. 

ERIC Clearinghouse for 
• r |3 Junior Colleges SEP ^ ^ 1991 

t*«fc«A'AS*>*A*.**A*A' A»A^A«A«A«/»***AA.*A*A*A*.* , A , A , ±*A' ****** 



